Samples

List S3 bucket and checkpoint

ListS3 -> Process -> Print

Partionning produced Data based on a String key (see meta extract partitionBy field)

Write in a s3 bucket

Write in a postgresql store

Write in an opensearch store

Activate the error SideStream

The error SideStream must currently point to a natsjs. The message produced must be then consumed from the topic configured. The error SideStream message structure is normalized and all error can be sent to the same topic.

Store the errors from SideStreams to Elasticsearch

Schedule a batch processing flow analytics with a cron schedule

Add a Sidestream

The status sidestream can be used to send arbitrary data to another function (process function or sink). This data should be used to provide information on the function processing status (eg: internal statuses of a multiple steps process). This should serve as a kind of status monitoring output, and might be changed in later release to match this need.

List S3 bucket and checkpoint​

ListS3 -> Process -> Print​

Partionning produced Data based on a String key (see meta extract partitionBy field)​

Write in a s3 bucket​

Write in a postgresql store​

Write in an opensearch store​

Activate the error SideStream​

Store the errors from SideStreams to Elasticsearch​

Schedule a batch processing flow analytics with a cron schedule​

Add a Sidestream​